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ABSTRACT 



This paper tries to set out a framework for thinking about 
whether the innovation known as "charter schools" is working. It is designed 
to help those who are designing evaluations and those who are thinking about 
accountability. The charter schools movement is a new state strategy for 
educational system change. Those who .ask whether the charter school is 
working must look at both innovations in the approach: the alternate sponsor 
and the school accountable for the results. To begin with, any examination of 
the charter school must examine the law, determining whether schools are 
actually being created and how good, and how different, the school designs 
being created really are. Other issues are the quality of sponsors' decisions 
about charter school approval and whether schools are responding with 
improvements of their own. How tough sponsors are about school improvement 
and how tough the state is about the sponsor's performance must be 
considered. A second set of questions arises about school performance: are 
the students learning?; what standards must be met?; and what measures of 
performance are being used? are all questions to consider. A final issue is 
determining what the school does about low performance. (SLD) 
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ClilTII f IIEIIS 

NATIONRl NE1W0RK 

conniocting and auppoiUng stato-lovel ctiartcr achool initiatives 



What does it mean to ask: "Is 'Charter 
Schools' Working?" 

Web editor's note: Not surprisingly, the charter school movement is coming under 
increasing scrutiny nationally, with news stories, documentaries, studies and other 
evaluations — of individual charter schools,., charter laws... the whole charter idea. 

This is natural. And, on balance this is good. Charters are all about accountability... to 
parents through choices they make on behalf of their children. ..to chartering authorities... to 
policy makers... and to taxpayers and the general public. 

Charter schools and charter supporters need to get ahead of this growing interest... to make 
sure they have a say in how the questions are framed. 

Ted Koldrie's latest contribution to this discussion is the following paper entitled, "What 
does it mean to ask: Is "charter schools" working'?" This important contribution to the 
"theology of the charter movement" is intended to stimulate your thinking and provoke your 
response. To register you views, e-mail Ted at ted&.charterfiiends. org . 



What does it mean to ask: "Is 'Charter Schools* Working?** 

By Ted Kolderie 

It's legitimate and important to ask whether charter schools' is working. 
States enacting charter laws and those advancing the charter idea have an 
obligation to answer. But to give an intelligent answer we must know what 
the question means. 

Unhappily there is some unclarity about this. The term "recharter schools" 
and the journalists focus on kids-in classrooms combine to draw attention 
away from 'charter' as an institutional innovation (which it is) and encourage 
people to think of it as a pedagogical innovation (which it is not). 

We are unlikely to have a useful discussion about whether t charter ' is 
working or about what to do if it isn't unless we clarify just what it is we're 
trying to evaluate. 

This paper tries to set out a framework for thinking about whether the 
innovation known as charter schools is working. It may be helpful to those 
designing evaluations and to those trying now to think about accountability. 
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Two dimensions: the Law, and the Schools 

Partly the charter idea is about schools. More important, though, it is a new 
state strategy for system-change. It is new, and it is a strategy, for the 
Legislature to open the way for some public body other than the local board 
to offer public education in the community (i.e., to sponsor charter schools). 

Those asking "Is it working" must look at both innovations: the alternate 
sponsor, and the school accountable for results. 

An evaluation of the charter program in a state needs to begin with an 
examination of the law. Some laws called "charter" do produce schools; some 
do not. The charter program in a state is not, in other words, simply an 
adding-up of how well in the aggregate the charter schools are working. 

A. How is the law working? 

If the law does not work at all there will not be schools at all. But to "work" 
the law has to produce good schools; and has to get rid of bad schools; and 
has to create dynamics that will cause districts to create better schools of their 
own. 

1 . Are schools in fact being created? 

Partly this is a simple matter of counting: If no schools are being created the 
law is clearly failing. But the evaluation should not stop there. Why no 
schools? Are there no proposals? Or are there proposals but no approvals? 

Success is more than just schools-appearing, however. If most of the schools 
being created (as in some states) are schools into which districts can off-load 
"at risk's or low-performing students, the law is not working. To be 'working' 
the program needs to produce schools also for regular students. Only if 
schools appear that are attractive to families the districts want to keep will 
districts respond with changes and improvements in their own programs. This 
systemic effect is a key part of the charter schools not working (see #5 
below.). 

2. How good tand how different) are the school designs? 

A 'charter-school' is not a kind of school; not itself a learning-design. It is an 
opportunity provided in law for teachers or others to bring in (or to create) 
some learning design. So the evaluation should ask: What are people doing 
with this opportunity? If those who organize the schools do (or go on doing) 
only conventional things in conventional ways, the law will not really be 
succeeding. If on the other hand high-quality designs for learning — and for 
governance and management — flow into the state increasingly through the 
charter program, the law is working. 
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A particular school may of course succeed by doing the same thing better 
than it is being done today in the district sector; and conversely a particular 
school may fail with a learning program it has brought into the commimity 
even if that program has been successful elsewhere. Here we have again the 
critical distinction between the law working and the school/s working. 

3. How good are sponsors decisions about approval? 

Quality is important, along with quantity. Good proposals are not enough: 
there must also be good decisions about the proposals. The better the designs 
and decisions 'upstream' the fewer problems there will be later. An evaluation 
should ask: What proportion of applications is approved? Are good designs 
approved and weak designs rejected, or vice versa? 

In most states where schools are appearing the Legislature has made it 
possible for more than one public body to be offering public education in the 
community. So in the important states there will be fascinating questions 
about the different behavior of the different kinds of sponsors. Do some give 
approvals more easily than others? Do some make better decisions than 
others do? Is there a difference between processes in which applications are 
batched up and reviewed competitively and processes in which applications 
are reviewed one-at-a-time, each on its own merits? Do districts when asked 
to sponsor make different decisions . . . display different attitudes . . . when 
they know the presence of an alternate sponsor ensures that a school will 
appear in the community even if the district says 'no'? 

4. Are districts responding with improvements of their own? 

The purpose of "charter schools" — of letting 'somebody else offer public 
education — is not just to create schools. 

For those who enact the laws it is to create dynamics that will cause the 
mainline district system to change and to improve. The charter schools, 
helpful as they may be to the students who enroll in them, are instrumental. 

The evaluation should ask whether districts do in fact act to improve their 
own programs in response to the appearance of charter laws and charter 
schools. Most evaluations so far have not looked for these second-order 
effects. To evaluate the 'ripple effect" requires looking simply at what 
districts do. It does not require asking why the districts are changing; does 
not require proving causation. It is enough to show the correlation. 

5. How tough are sponsors about school Performance? 

Charter schools unlike most other schools have a dual accountability; to 
parents, as schools of choice, and to public authority, directly. An evaluation 
should look at whether parents stay, or leave. Do more want to enroll? How 
long is the waiting-list? How are the lapse provisions about eligibility 
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working? Were the schools opened equally to any student living in the state? 
Have limitations or preferences skewed the makeup of the student bodies? 

The sponsor is responsible for the school's performance. (Not for its day-to- 
day management, which is the responsibility of the school's own board.) The 
laws impose a duty on sponsors to oversee performance. So a key question is 
how and how well the sponsor/s monitor performance. And, of course, how 
decisively the sponsors act when they find performance falling short. Are 
schools that need to be closed in fact closed, with their charters either 
revoked during the course of their term or not renewed at the end of their 
term? 

Charter schools freedom from process-controls is to be a tradeoff for their 
real accountability for results. The reality must match the theory. States will 
evaluate the sponsors by evaluating how well the sponsors oversee and hold- 
accountable their schools. A failure in student learning is a failure of the 
school. A failure to deal with that failure of the school is a failure of the law. 

6. How tough is the state about sponsor's performance? 

The Legislature that wrote the charter law is responsible for making any 
changes needed in the law. Some states (the Office of Legislative Auditor in 
Minnesota comes to mind) have general authority to do program evaluation. 
Some state charter laws contain specific requirements for periodic evaluation. 
A pattern of annual changes and improvements is already visible in some 
states with strong laws. 

Legislatures mostly react to proposals. Individual legislators may propose 
some changes . . . corrections and improvements. But most proposals are 
likely to come from others. 

As a charter program grows the school-operators, through their association, 
will normally go back to the state with suggestions for change. A special 
obligation falls on the '"fnends" of the charter program in a state to monitor 
both schools and sponsors, and to propose changes where the law is not 
working well. 

An absolutely critical element of a successful charter program, therefore, is 
the development of a support group or resource center to work for quality 
proposals, quality approvals, quality schools with sound learning programs 
and competent management; insisting, in the interests of the charter idea, on 
rigorous accountability both for schools and for students in the charter public 
system. (The appearance of such support systems is an important new 
development that should be understood by those who ask, "Is 'charter schools' 
working?) 



B. How are the schools working? 

This is mainly but not entirely the question, "Are the students learning?" It is 
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necessary also for a school to succeed financially and in its relationships with 
its parents and its teachers. In the years before the program has had time to 
have a measurable effect on student learning, these other dimensions may be 
the most important; telling us, in effect, whether the school holds real 
potential for student learning or not. 

On both dimensions it will be important to give the school time to show 
results. It is reasonable to expect some false starts, some failures, in the early 
years. Changes in personnel or in school practice may in fact be a sign the 
school is learning from its mistakes; is working. 

With respect to student learning the hardest thing for evaluators (and for 
journalists) may be to remember that a charter school, unlike so many 
education innovations, is not a learning design. It is not possible as a result to 
link charter schools and student-learning directly. The evaluation of student 
learning has to proceed from an examination of the particular learning 
program/s in use. 

1. What standards does the school have to meet? 



Certain learning designs may come with standards specific to that design. In 
some states (some) standards may be set in law; imposed on charter schools 
as on all public schools. Some other additional standards may be set in the 
charter itself, by the sponsor. Some standards, finally, may be set by the 
school even if not required by the law or by the charter. Some will be 
academic. Some may (perhaps should) establish other, different dimensions 
of a school s success or failure. 



It is important to recognize that this country is nowhere near clear with itself 
yet about what young people should know and be able to do, and how well. It 
is not a simple matter of whether or not to have standards, supported by right- 
minded people and opposed by the wrong-minded. It involves a serious of 
question about how we define education. And of course about whose 
definition should prevail. 

2. What measures of performance are being used? 

Again: Some measurements may come with a particular learning design. 
Some tests/measurements will be required by the state (With, or apart from, 
state standards). Some additional measurements may be required by the 
sponsor; written into the charter. And some measurements of performance 
may be undertaken (even, developed) by the school itself, or by charter 
schools in a state collectively . . . especially if the school believes the tests 
required by state/sponsor do not well measure what the school is trying to 
accomplish and is in fact accomplishing. 



Tests are powerful. They can be dangerous. Especially where a lot is at stake 
teachers . . . schools . . . will often teach to the test. Tests not aligned with 
standards can draw schools away from their real purposes. A charter school 
may be entitled to an assessment aligned with its own defined purposes. 
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3. What does the school do about low Performance? 



The responsibility for the successful operation of the charter school falls on 
the board of the school. It has the duty to deal with problems that arise: in the 
academic work, in the fiscal management and in the administration. It also 
has a strong incentive to do so, since it knows its existence depends on its 
success with its students and with its management. 

It will require a new discipline on the part of districts (where they are 
sponsors) to leave 'process' to the school and to focus on results; and, where 
results do fall short, to enforce consequences rather than try to fix' the 
school. 

Fixing the school is responsibility of the school. The evaluation should ask 
what those responsible for the school does when performance in the school — 
student performance or administrative/fiscal performance — falls short. What 
does the school director do? What does the board of the school do? How 
decisively do they act? What consequences do they impose? On whom? On 
students? On teachers? On administrators? What constructive improvements 
are made in teaching methods, or in administration and management? 

If the board of the school does not act effectively then, as discussed above, 
the sponsor should revoke or not-renew. 

The charter idea depends on sponsors acting courageously to enforce 
accountability. If they do, and if schools know they do, then accountability 
should work at the school level. If the sponsors fail then the law is failing and 
the Legislature should change the law. 
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